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ABSTRACT 

Contents of this report from Florida on the National 
Council of Teachers of English achievement awards in writing are 
divided into seven sections: the principles followed in pairing the 
twenty-two judges into eleven teams; the principles followed in 
assigning students* papers to judges; the procedure used in selecting 
winners; the range of students* performances (with a table and a 
graph); a comparison of ratings for impromptu, autobiographical, and 
best writing papers (with a table) ; the range o differences in 
judges* evaluations (with a table) ; and a comparison of ratings by: 
(1) public school teachers, and (2) junior college and university 
teachers (with a table) . (JH) 
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Part 1 1 Principles followed In pairing the twenty-two .judges into eleven teams 

--The two judges In a team were from different sections of the state. 

—In almost every case one judge from each team was from the public schools 
and the other was from a junior college or university. (There were two 
partial exceptions to this principle. One team was made up of two judges 
from junior colleges, but one of the two judges has spent most of her 
career in public school teaching. Another team was made up of two people 
who have spent their careers in public school teaching, though one of them 
had just accepted a position in a junior college.) 



Part Hi Principles followed in assigning students 1 papers to Judges 

—No judge read a paper from his or her school or immediate locality. 

—Usually the papers of only one student from a particular school went to a 
judging team. In some cases, when avoiding other problems made it 
necessary, the papers of two students from the same school went to a 
judging team. 

--The Judges did not know what schools their papers were from. 

—Each team judged approximately the same number of papers. (Ten teams 
had eighteen sets of papers each? one team had nineteen sets.) 



Part HIi Procedure used in selecting winners 
—Florida could have as many as 30 winners. 

—17 students were chosen as winners by both judges { these were named winners 
without any further study of their papers. The range of these students 1 
scores was from 19 to 22. (The highest possible score was 24—12 points 
from each judge.) (The range includes twelve studentsi five of these 
seventeen students did not receive specific scores from one of their judges.) 

—47 students were chosen as winners by only one judge. The range of these 
students 1 scores was from 13 to 21, (This range includes all forty-seven 
students} the papers of nine students who received specif io scores from 
only one of their two judges were rated with specific scores by the state 
coordinator. ) 

—9 of the remaining 11 positions went to students (chosen as winners by 
one judge) with scores of 20 and El without any further study cf their 
papers, 

—12 students had scores of 19. The state coordinator studied the papers 
of these twelve students and chose as winners the four whose work he thought 
was beat. This action filled the remaining 4 placesi 1? + 9 + 4 « 30. 
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Part IV i Range of students 1 performance (See graph on next page,) 

Keyt Score of 3 represents "undistinguished writing." 

Score of 6 represents "good writing, but no better than might be 

expected of a nominee." 
Score of 2 represents "unusually effective writing." 
Score of 12 represents "outstanding writing." 

Notes i The study is based on the writing of 199 students. 

There are two scores for each of 163 students, only one score for 
each of 36 students. There are, then, 362 scores, 

Each score represents the accumulative rating for the student's 

three papers— an impromptu theme, an autobiographical essay, and a sample 

of his or her best writing. A top score of 4 is possible on each paper. 



Number of Per (tent of Instances 



Score 


Instances 




the Score Occurred 


3 


10 




2.76 % 








6.63 % 


5 


39 




10.77 % 


6 


58 


a 


16.02 % 
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62 
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17.13 % 
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48 




13.26 % 
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12.98 % 


10 


38 
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10.50 % 


11 


30 




8.29 % 


1Z 


6* 


S3 


1.66 % 



^Though six students received a rating of 12 by one of the two judges, 
no student received a rating of 12 by both judges. 
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Part V« Comparison of ratings for Impromptu. Autobiographical and Best Writing Papers 

Key i Score of 1 represents "undistinguished paper." 

Score of 2 represents "good paper , but no better than might be expected 
of nominee." 

Score of 3 represents "unusually effective paper." 
Score of % represents "outstanding paper." 

Note i The averages (means) are based on 362 ratings for each of the kinds of papers. 



I 

I 

Kind of Paper * Mean Spore 

Impromptu •••••« 2,32 

Autobiographical 2 ,32 

Best Writing 2^8 



*The Impromptu was a one-hour paper on a subject assigned by NCTE. The 
Autobiographical paper was a short statement (25Q-O00 words recommended) written 
at the student's leisure, The student and his or her teacher could select 
the Best Writing sample; no genre was specified. 



Part VI. i Range of differences In Judges 1 evaluations 

This part of the study is based on the judges' ratings of the 163 students 
for whom there is a rating from each of the two judges. A judge could award 
a student's writing a total rating of from 2 to 12. Tne possibility exists, 
then, that the two judges could differ by as much as 9 points on a lft-point 
scale. The following table shows the difference in total scores that the 
judges assigned to a student's papers. (The 0 indicates that the two judges 
awarded the same scores to the student's writing.) 

Difference in 



Two Judges' Number of Per Cent of 

Ratings Instances Instances 

0 25 15.3^ * 

1 58 35.58 * 

2 kl 25.15 % 

3 23 14.11 % 
k 13 7.98 % 

5 1 0.61 % 

6 2 - ■ 1.23 % 
7, 8, 9 0 =- 0.00 % 



One way to interpret this data is to consider that there are k descriptive 
classifications of the students' writing and 10 possible ratings (from 3 through 12, 
inclusive), A descriptive classification, then, covers a range of 2-f numbers* 
This being true, we can say that in from 76 % to 90 % of the cases the two judges' 
ratings fell within the range of one descriptive classification. 
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Ratings 



Key i Rating of ^ represents "undistinguished writing." 

Rating of 6 represents "good writing, but no better .than might 
be expected of a nominee." 

Rating of 2 represents "unusually effective writing," 

Rating of 12 represents "outstanding writing." 

Notes t The study is based on the writing of 199 students. 

There are two ratings for each of 163 students, only one rating 
for each of 36 students. There are, then, 362 ratings. 

Each rating represents the total rating for the student three 
papers— an impromptu theme, an autobiographical essay, 
and a sample of his or her best writing, A top score of 
4 is possible on each paper. 
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Part VII i Comparison of ratings by (l)publlc school teachers and (2) junior 
college and university teachers 

This part of the data Is based on the evaluations by seven teams of 
judges. (Data is not used for four of the teams because specific numerical 
scores are lacking from one judge in each of two teams and since in two 
teams the judges were not paired to clearly represent both (l)the public 
schools and (2)the junior colleges and universities.) These seven teams 
of judges evaluated the writing of 127 students. 

The table below shows each judge's averge (mean) rating for the 18 or 19 
students whose papers he evaluated. The lowest possible rating for a 
student is J[j the highest, 12. 





Junior College 
or University 
Judge 


Public 
School 
Judge 


Difference 
in 

Averages 


Team A 


8.1J4 


8.33 


.11 


Team B 


7.63 


8.11 


.48 


Team C 


6.61 


5.33 


1.28 


Team D 


8.11 


7.39 


.72 


Team E 


5.67 


5.50 


.17 


Team P 


7.00 


7.05 


.05 


Team G 


6.39 


6.56 


.17 


Average 


7.12 


6.90 


.22 



On a 10-point scale Q through 12, inclusive) the judges' averages differed less 
than .5 of a point in five teams « In one team they differed between .5 and 
1 point. In one team they differed between 1 and 1*5 points. In three teams 
the public school judges awarded the higher scores) In four teams the junior 
college or university judges awarded the higher scores. The average score 
awarded by the junior college and university judges was slightly higher than 
the average score awarded by public school judges. 



Final note on similarity and difference in judges' conclusions i all 11 teams 
and 22 judges and all 199 students are included In these statements. 

—17 students were chosen as winners by both judges. 

—47 students were chosen as winners by one judge but not the other. 

— 135 students were chosen as winners by neither judge. 
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